Textured digital elevation model generation

ABSTRACT

For textured digital elevation model generation, a method projects swath point projection coordinates of each lidar point of texel swaths into a corresponding texel swath with normalized projection coordinates. Each texel swath includes lidar points, an optical image, and swath output projection point coordinates of the lidar points into the optical image. The method generates a projection matrix including lidar point data for each lidar point of each texel swath. The method calculates enhanced lidar coordinates for each lidar point as a function of minimizing misregistration errors. In addition, the method creates enhanced output pixel coordinates for each image pixel based on the enhanced lidar coordinates for the lidar points and the associated output pixel coordinates. The method generates a textured digital elevation model based on the enhanced output pixel coordinates for each image pixel, and the enhanced lidar coordinates.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/326,158 entitled “TEXTURED DIGITAL ELEVATION MODEL FORMATION” andfiled on Apr. 22, 2016 for Taylor C. Bybee, which is incorporated hereinby reference.

BACKGROUND

Field

The subject matter disclosed herein relates to textured digitalelevation model generation.

Description of the Related Art

Lidar sensors and optical sensors may collect elevation data in multipleswaths.

BRIEF SUMMARY

A method for textured digital elevation model generation is disclosed.The method projects swath point projection coordinates of each lidarpoint of texel swaths into a corresponding texel swath with normalizedprojection coordinates. Each texel swath comprises lidar points, anoptical image, and swath output projection point coordinates of thelidar points into the optical image. The method generates a projectionmatrix comprising lidar point data for each lidar point of each texelswath. The lidar point data comprises the normalized projectioncoordinates. The method calculates enhanced lidar coordinates for eachlidar point as a function of minimizing misregistration errors. Inaddition, the method creates enhanced output pixel coordinates for eachimage pixel based on the enhanced lidar coordinates for the lidar pointsand the associated output pixel coordinates. The method generates atextured digital elevation model based on the enhanced output pixelcoordinates for each image pixel, and the enhanced lidar coordinates. Anapparatus and program product also perform the functions of the method.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of the embodiments of the invention will bereadily understood, a more particular description of the embodimentsbriefly described above will be rendered by reference to specificembodiments that are illustrated in the appended drawings. Understandingthat these drawings depict only some embodiments and are not thereforeto be considered to be limiting of scope, the embodiments will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings, in which:

FIG. 1 is a schematic drawing illustrating one embodiment of a texelswath capture system;

FIG. 2A is a schematic drawing illustrating one embodiment of a Texelswath;

FIG. 2B is a schematic drawing illustrating one embodiment ofoverlapping texel swaths;

FIG. 2C is a schematic drawing illustrating one embodiment ofidentifying misregistration errors;

FIG. 2D is a drawing illustrating one embodiment of coordinate spaces;

FIG. 3A is a schematic block diagram illustrating one embodiment of aswath database;

FIG. 3B is a schematic block diagram illustrating one embodiment ofswath data;

FIG. 3C is a schematic block diagram illustrating one embodiment oflidar point data;

FIG. 3D is a schematic block diagram illustrating one embodiment of aprojection matrix;

FIG. 3E is a schematic block diagram illustrating one embodiment oftextured digital elevation model;

FIG. 3F is a schematic block diagram illustrating one embodiment ofpixel data;

FIG. 4 is a schematic block diagram illustrating one embodiment of acomputer; and

FIGS. 5A-B are schematic block diagrams illustrating one embodiment of atextured digital elevation model generation method.

DETAILED DESCRIPTION OF THE INVENTION

As will be appreciated by one skilled in the art, aspects of theembodiments may be embodied as a system, method or program product.Accordingly, embodiments may take the form of an entirely hardwareembodiment, an entirely software embodiment (including firmware,resident software, micro-code, etc.) or an embodiment combining softwareand hardware aspects that may all generally be referred to herein as a“circuit,” “module” or “system.” Furthermore, embodiments may take theform of a program product embodied in one or more computer readablestorage devices storing machine readable code, computer readable code,and/or program code, referred hereafter as code. The storage devices maybe tangible, non-transitory, and/or non-transmission. The storagedevices may not embody signals. In a certain embodiment, the storagedevices only employ signals for accessing code.

Many of the functional units described in this specification have beenlabeled as modules, in order to more particularly emphasize theirimplementation independence. For example, a module may be implemented asa hardware circuit comprising custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors, or otherdiscrete components. A module may also be implemented in programmablehardware devices such as field programmable gate arrays, programmablearray logic, programmable logic devices or the like.

Modules may also be implemented in code and/or software for execution byvarious types of processors. An identified module of code may, forinstance, comprise one or more physical or logical blocks of executablecode which may, for instance, be organized as an object, procedure, orfunction. Nevertheless, the executables of an identified module need notbe physically located together, but may comprise disparate instructionsstored in different locations which, when joined logically together,comprise the module and achieve the stated purpose for the module.

Indeed, a module of code may be a single instruction, or manyinstructions, and may even be distributed over several different codesegments, among different programs, and across several memory devices.Similarly, operational data may be identified and illustrated hereinwithin modules, and may be embodied in any suitable form and organizedwithin any suitable type of data structure. The operational data may becollected as a single data set, or may be distributed over differentlocations including over different computer readable storage devices.Where a module or portions of a module are implemented in software, thesoftware portions are stored on one or more computer readable storagedevices.

Any combination of one or more computer readable medium may be utilized.The computer readable medium may be a computer readable storage medium.The computer readable storage medium may be a storage device storing thecode. The storage device may be, for example, but not limited to, anelectronic, magnetic, optical, electromagnetic, infrared, holographic,micromechanical, or semiconductor system, apparatus, or device, or anysuitable combination of the foregoing.

More specific examples (a non-exhaustive list) of the storage devicewould include the following: an electrical connection having one or morewires, a portable computer diskette, a hard disk, a random-access memory(RAM), a read-only memory (ROM), an erasable programmable read-onlymemory (EPROM or Flash memory), a portable compact disc read-only memory(CD-ROM), an optical storage device, a magnetic storage device, or anysuitable combination of the foregoing. In the context of this document,a computer readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

Code for carrying out operations for embodiments may be written in anycombination of one or more programming languages, including anobject-oriented programming language such as Python, Ruby, Java,Smalltalk, C++ or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The code may execute entirely on the user's computer, partlyon the user's computer, as a stand-alone software package, partly on theuser's computer and partly on a remote computer or entirely on theremote computer or server. In the latter scenario, the remote computermay be connected to the user's computer through any type of network,including a local area network (LAN) or a wide area network (WAN), orthe connection may be made to an external computer (for example, throughthe Internet using an Internet Service Provider).

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment. Thus, appearances of the phrases“in one embodiment,” “in an embodiment,” and similar language throughoutthis specification may, but do not necessarily, all refer to the sameembodiment, but mean “one or more but not all embodiments” unlessexpressly specified otherwise. The terms “including,” “comprising,”“having,” and variations thereof mean “including but not limited to,”unless expressly specified otherwise. An enumerated listing of itemsdoes not imply that any or all of the items are mutually exclusive,unless expressly specified otherwise. The terms “a,” “an,” and “the”also refer to “one or more” unless expressly specified otherwise.

Furthermore, the described features, structures, or characteristics ofthe embodiments may be combined in any suitable manner. In the followingdescription, numerous specific details are provided, such as examples ofprogramming, software modules, user selections, network transactions,database queries, database structures, hardware modules, hardwarecircuits, hardware chips, etc., to provide a thorough understanding ofembodiments. One skilled in the relevant art will recognize, however,that embodiments may be practiced without one or more of the specificdetails, or with other methods, components, materials, and so forth. Inother instances, well-known structures, materials, or operations are notshown or described in detail to avoid obscuring aspects of anembodiment.

Aspects of the embodiments are described below with reference toschematic flowchart diagrams and/or schematic block diagrams of methods,apparatuses, systems, and program products according to embodiments. Itwill be understood that each block of the schematic flowchart diagramsand/or schematic block diagrams, and combinations of blocks in theschematic flowchart diagrams and/or schematic block diagrams, can beimplemented by code. This code may be provided to a processor of ageneral-purpose computer, special purpose computer, or otherprogrammable data processing apparatus to produce a machine, such thatthe instructions, which execute via the processor of the computer orother programmable data processing apparatus, create means forimplementing the functions/acts specified in the schematic flowchartdiagrams and/or schematic block diagrams block or blocks.

The code may also be stored in a storage device that can direct acomputer, other programmable data processing apparatus, or other devicesto function in a particular manner, such that the instructions stored inthe storage device produce an article of manufacture includinginstructions which implement the function/act specified in the schematicflowchart diagrams and/or schematic block diagrams block or blocks.

The code may also be loaded onto a computer, other programmable dataprocessing apparatus, or other devices to cause a series of operationalsteps to be performed on the computer, other programmable apparatus orother devices to produce a computer implemented process such that thecode which execute on the computer or other programmable apparatusprovide processes for implementing the functions/acts specified in theflowchart and/or block diagram block or blocks.

The schematic flowchart diagrams and/or schematic block diagrams in theFigures illustrate the architecture, functionality, and operation ofpossible implementations of apparatuses, systems, methods and programproducts according to various embodiments. In this regard, each block inthe schematic flowchart diagrams and/or schematic block diagrams mayrepresent a module, segment, or portion of code, which comprises one ormore executable instructions of the code for implementing the specifiedlogical function(s).

It should also be noted that, in some alternative implementations, thefunctions noted in the block may occur out of the order noted in theFigures. For example, two blocks shown in succession may, in fact, beexecuted substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved. Other steps and methods may be conceived that are equivalentin function, logic, or effect to one or more blocks, or portionsthereof, of the illustrated Figures.

Although various arrow types and line types may be employed in theflowchart and/or block diagrams, they are understood not to limit thescope of the corresponding embodiments. Indeed, some arrows or otherconnectors may be used to indicate only the logical flow of the depictedembodiment. For instance, an arrow may indicate a waiting or monitoringperiod of unspecified duration between enumerated steps of the depictedembodiment. It will also be noted that each block of the block diagramsand/or flowchart diagrams, and combinations of blocks in the blockdiagrams and/or flowchart diagrams, can be implemented by specialpurpose hardware-based systems that perform the specified functions oracts, or combinations of special purpose hardware and code.

Bybee, Taylor C., and Budge, Scott E., “Textured Digital Elevation ModelFormation from Low-cost UAV Lidar/Digital Image Data;” and Bybee,Taylor, “An Automatic Algorithm for Textured Digital Elevation ModelFormation Using Aerial Texel Swaths” are incorporated herein byreference.

The description of elements in each figure may refer to elements ofproceeding figures. Like numbers refer to like elements in all figures,including alternate embodiments of like elements.

FIG. 1 is a schematic drawing illustrating one embodiment of a texelswath capture system 100. The system 100 may capture texel swaths 160 ofa physical terrain 120. The texel swaths 160 are used to generate atextured digital elevation model. In the depicted embodiment, the systemincludes an aerial platform 115. The aerial platform 115 may be anautonomous vehicle, a drone, an aircraft, and the like.

A lidar sensor 105 and one or more optical sensor 110 may be mounted onthe aerial platform 115. The lidar sensor 105 may capture a plurality oflidar points 140 for a texel swath 160. Each optical sensor 110 maycapture an optical image 145 for the texel swath 160. The one or moreoptical sensors 110 may include a visible spectrum optical sensor 110,an infrared optical sensor 110, and the like. The combination of thelidar points 140 and the one or more optical images 145 form the texelswath 160. The lidar points 140 and the optical images 145 may be storedon a swath database 135.

In one embodiment, the system 100 includes a Global Positioning System(GPS) 125, a gyroscope, and/or an accelerometer 130. The GPS 125 andaccelerometer 130 may generate pose values that specifies the positionand attitude of the lidar sensor 105 and the optical sensor 110.

The texel swaths 160 may be captured from different altitudes. Inaddition, image points 310 within the texel swaths 160 may be capturedfrom different sensor attitudes. Furthermore, the paths of the aerialplatform 115 for each of the texel swaths 160 may not be parallel. Thepose values for the texel swaths may also include measurement errors.

The embodiments described herein generate a textured digital elevationmodel by calculating enhanced lidar coordinates for each lidar point 140as a function of minimizing misregistration errors, creating enhancedoutput pixel coordinates of each pixel of the optical image 145 based onthe enhanced lidar coordinates for the lidar points 140 and associatedoutput pixel coordinates, and generating the textured digital elevationmodel based on the enhanced output pixel coordinates and enhanced lidarcoordinates as will be described hereafter.

FIG. 2A is a schematic drawing illustrating one embodiment of the texelswath 160. The texel swath 160 includes one or more lidar points 140.The lidar points 140 may be fused with the optical image 145. Forsimplicity, the embodiments will be described for the case of a singleoptical image 145. However, the embodiments may be performed for each ofthe one or more optical images 145. The optical image 145 may include aplurality of image pixels 165. In one embodiment, each lidar point 140is fused to a corresponding optical image 145 at a calibrated pixellocation.

FIG. 2B is a schematic drawing illustrating one embodiment ofoverlapping texel swaths 160 a-c. In the depicted embodiment, threetexel swaths 160 a-c overlap. Any number of texel swaths 160 may beemployed. As shown, two or more pluralities of lidar points 140 overlapeach texel swath 160.

FIG. 2C is a schematic drawing illustrating one embodiment ofidentifying misregistration errors. A lidar point 140 is capturedconcurrently with a first optical image 145 a in the first texel swath160 at first coordinates X_(ij) _(x) , X_(ij) _(y) , X_(ij) _(xz) 157.The lidar point 140 is also in a field of view of a second optical image145 b in a second texel swath 160. The range of the lidar point 140 isnot measured for the second optical image 145 b. To calculate enhancedlidar coordinates for the original lidar point 140, the original lidarpoint 140 may be transformed from the first coordinates 157 to new lidarcoordinates 156. First matching normalized projection coordinates 231 aof the first coordinates 157 in the first optical image 145 a correlateswith second matching normalized projection coordinates 231 b of thelidar coordinates 157 in the second optical image 145 b. The normalizedpoint projections x_(ij), y_(ij) 286/289 may be calculated usingEquation 1 from the new lidar coordinates X_(ij) _(x) , X_(ij) _(y) ,X_(ij) _(xz) 156 in the respective capture coordinates spaces of thefirst or second texel swaths 160, wherein the first point normalizedprojection 286 is a projection of the new lidar coordinates 156 in thefirst optical image 145 a and the second normalized point projection 289is a projection of the new lidar coordinates 156 into the second opticalimage 145 b.

$\begin{matrix}{\begin{bmatrix}x_{il} \\y_{ij}\end{bmatrix} = \begin{bmatrix}\frac{X_{{ij}_{x}}}{X_{{ij}_{z}}} \\\frac{X_{{ij}_{y}}}{X_{{ij}_{z}}}\end{bmatrix}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

The misregistration errors include a range error 288, a calibratedprojection error 275, and a matching projection error 280. The rangeerror 288 may be a difference of distances between a first vector 287 tothe first lidar coordinate 157 and a second vector 285 to the new lidarcoordinates 156. The first vector 287 may be the measured range 290. Thecalibrated projection error 275 may be a distance on a normalize imageplane between the first matching normalized projection coordinate 231 aand the first normalized point projection 286. The matching projectionerror 280 may be a distance from the second normalized point projection289 to the second matching normalized projection coordinate 231 b. Inone embodiment, the vectors 285/287 may be calculated using Equation 2.λ_(ij)=√{square root over (X _(ij) _(x) ² +X _(ij) _(y) ² +X _(ij) _(z)²)}  Equation 2

FIG. 2D is a drawing illustrating one embodiment of coordinate spaces.

The coordinate spaces include a terrain coordinate space 305, a capturecoordinate space 310, and an output coordinate space 315. The terraincoordinate space 305 may be coordinate space for the terrain 120. Thecapture coordinate space 310 may be the coordinate space in which theoptical image 145 and the lidar points 140 are captured. The outputcoordinate space 315 may be the coordinate space of a textured digitalelevation model. Each of the coordinate spaces 305/310/315 may beindependent and may be related by one or more transformations.

FIG. 3A is a schematic block diagram illustrating one embodiment of theswath database 135. The swath database 135 may be organized as a datastructure in a memory. In the depicted embodiment, the swath database135 includes a plurality of instances of swath data 200.

FIG. 3B is a schematic block diagram illustrating one embodiment of theswath data 200. In the depicted embodiment, the swath data 200 includesa swath index 205, optical image pixels 210 for one or more opticalimages 145, swath pixel coordinates 213, swath point projection pixelcoordinates 217, lidar point data 255, pose values 220, a swathcoordinate transformation 225, matching projection coordinates 228,matching image pixels 229, output pixel coordinates 230, and enhancedoutput pixel coordinates 235.

The swath index 205 may uniquely identify the swath 160. The opticalimage pixels 210 may store the original image pixels 165 of the opticalimage 145. The swath pixel coordinates 213 may record the twodimensional (2D) coordinates of the optical image pixels 210 in thecapture coordinate space 310. The swath point projection pixelcoordinates 217 may be 2D pixel coordinates of a swath point projection.The swath data 200 may include multiple instances of lidar point data255. The lidar point data 255 is described in more detail in FIG. 3C.

The pose values 220 may record a position and/or attitude of the lidarsensor 105 and the optical sensors 110 while capturing the texel swath160. In one embodiment, the attitude and position of a pose value a_(j)220 for a given lidar sensor j 105 and/or optical sensor j 110 isrepresented as a vector comprising a quaternion describing the attitudeq_(j0), q_(j1), q_(j2), q_(j3) concatenated with coordinates of theposition t_(jx), t_(jy), t_(jz), as shown in Equation 3.a _(j) =[q _(j0) ,q _(j1) ,q _(j2) ,q _(j3) ,t _(jx) ,t _(jy) ,t_(jz)]^(T)  Equation 3

The output pixel coordinates 230 may be the 2D pixel coordinates for thelidar point 140 in the output coordinate space 315. The swath coordinatetransformation 225 may transform the swath pixel coordinates 213 for alidar point 140 or an image pixel 165 into corresponding 2D output pixelcoordinates 230 in the output coordinate space 315. The matchingprojection coordinates 228 may be 2D coordinates of a matching opticalimage pixel 210 on a normalized image plane of a swath 160. The matchingimage pixels 229 may be image pixels 165 in the optical image pixels 210of a texel swath 160 that match optical image pixels 210 in an adjacentswath 160. The output pixel coordinates 230 may be enhanced as theenhanced output pixel coordinates 235. The enhanced output pixelcoordinates 235 may be based on enhanced lidar coordinates for one ormore lidar points 140 as will be described hereafter. The enhancedoutput pixel coordinates 235 may be used to generate the textureddigital elevation model.

FIG. 3C is a schematic block diagram illustrating one embodiment of thelidar point data 255. In the depicted embodiment, the lidar point data255 includes a lidar point index 465, lidar point coordinates 455, swathpoint projection coordinates 453, swath output point projectioncoordinates 457, swath pixel coordinates 460, output point projectioncoordinates 462, output lidar point coordinates 463, enhanced lidarcoordinates 270, enhanced output point projection coordinates 472, thenormalized projection coordinates 155, calibrated projection coordinates159, the calibrated projection error 275, the matching projection error280, and the range error 288. In a certain embodiment, the lidar pointdata 255 includes a range measurement 290. The range measurement 290 mayonly be stored if available.

The lidar point index 465 may uniquely identify the lidar point 140. Thelidar point coordinates 455 may be the original three dimensional (3D)coordinates recorded by the lidar sensor 110 in the capture coordinatespace 310. The swath point projection coordinates 453 may be 2Dprojection coordinates of the lidar point 140 into a swath 160. Theswath output point projection coordinates 457 may be 2D coordinates ofthe lidar point coordinates 455 transformed into the output coordinatespace 315. The swath pixel coordinates 460 may be the corresponding 2Dswath pixel coordinates 213 for the lidar point coordinates 455 for agiven texel swath 160. The output lidar point coordinates may be 3Dpoint coordinates of the lidar point 140 when output into the outputcoordinate space 315. The output point projection coordinates 462 may bea 2D projection of output lidar point coordinates 463 into an outputimage in the output coordinate space 315.

The enhanced lidar coordinates 270 may be coordinates for the lidarpoint 140 in the output coordinate space 315. The output pointprojection pixel coordinates 462 may be coordinates of the output lidarpoint coordinates 463 projected into the output image. The normalizedprojection coordinates 155 may be 2D coordinates of a projection of alidar point 140 into a normalized image plane of a texel swath 160. Thecalibrated projected coordinates 159 may be 2D coordinates of aprojection of transformed lidar point 140 into a normalized image planeof a texel swath 160.

The determination of the calibrated projection error 275, matchingprojection error 280, and range error 288 is shown in FIG. 2C. The rangemeasurement 290 may be captured as part of capturing the lidar point140.

FIG. 3D is a schematic block diagram illustrating one embodiment of aprojection matrix 250. The projection matrix 250 may organize theplurality of lidar point data 255. The projection matrix 250 may beorganized as a data structure in a memory. In the depicted embodiment,the projection matrix 250 includes lidar point data rows 260, whereineach lidar point data row 260 includes the lidar point data 255 for agiven lidar point 140 in each texel swath 160. The projection matrix 250further includes swath columns 265, wherein each swath column 265includes all potential lidar point data 255 in a given texel swath 160.As each texel swath 160 does not include all lidar points 140, theprojection matrix 250 may be organized as a banded sparse matrix. Theorganization of the projection matrix 250 allows the enhanced lidarcoordinates 270 to be calculated more efficiently by organizing thelidar point data 255 separately from the swath data 200 and in thesparse matrix for efficient calculation.

FIG. 3E is a schematic block diagram illustrating one embodiment of thetextured digital elevation model 425. The textured digital elevationmodel 425 may represent the terrain 120. The textured digital elevationmodule 425 may be organized as a data structure in a memory. In thedepicted embodiment, the textured digital elevation module 425 includesa plurality of output pixels 430. Each output pixel 430 may correspondto one or more image pixels 165 of at least one texel swath 160.

FIG. 3F is a schematic block diagram illustrating one embodiment of theoutput pixel 430. The output pixel 430 may be used to render a pixel ofthe textured digital elevation module 425. In the depicted embodiment,the output pixel 430 includes pixel data 435 for an image pixel 165 ofthe optical image pixels 210 and the corresponding enhanced output pixelcoordinates 235. In addition, the output pixel 430 includes thecorresponding enhanced lidar coordinates 270. The organization of thepixel data 435, the enhanced output pixel coordinates 235, and theenhanced lidar coordinates 270 supports efficient calculation of thetextured digital elevation model 425.

FIG. 4 is a schematic block diagram illustrating one embodiment of acomputer 400. In the depicted embodiment, the computer 400 includes aprocessor 405, a memory 410, and communication hardware 415. The memory410 may a semiconductor storage device, a hard disk drive, a holographicstorage device, a micromechanical storage device, or combinationsthereof. The memory 410 may store code. The processor 405 may executethe code. The communication hardware 415 may communicate with otherdevices.

FIGS. 5A-B are schematic block diagrams illustrating one embodiment of atextured digital elevation model generation method 500. The method 500generates a textured digital elevation model 425. The method 500 may beperformed by the system 100, the computer 400, and/or the processor 405.

The method 500 starts, and in one embodiment, the system 100 calibrates503 the optical image 145 captured by the optical sensor 110. Thecalibration may remove lens distortion at one or more image pixels 165.

In one embodiment, the system 100 captures 505 two or more texel swaths160. Each texel swath 160 may comprise lidar points 140, an opticalimage 145, and/or swath output projection point coordinates 457 of thelidar points 140 into the optical image 145. The optical sensor 110 maycapture 505 the optical image 145 and store the optical image 145 asswath data 200. In addition, the lidar sensor 105 may capture 505 thelidar points 140 and store the lidar points 140 to the swath database135 as corresponding lidar point data 255 in the swath data 200. Thetexel swaths 160 may be captured 505 in the capture coordinate space310.

The processor 405 may transform 510 the lidar point coordinates 455 ofeach lidar point 140 from the texel swaths 160 into the swath outputpoint projection coordinates SPPC 457 of the output coordinate space315. The lidar point coordinates LPPC 455 may be transformed as shown inEquation 4, where M_(A) is a transformation matrix.SPPC=M _(A)LPPC  Equation 4

One embodiment of M_(A) is shown in Equation 5.

$\begin{matrix}{M_{A} = \begin{bmatrix}{1 - \frac{2\left( {q_{j_{2\;}}^{2} + q_{j_{3}}^{2}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}}} & \frac{2\left( {{q_{j_{1}}q_{j_{2\;}}} - {q_{j_{0}}q_{j_{3}}}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}} & \frac{2\left( {{q_{j_{1}}q_{j_{3\;}}} + {q_{j_{0}}q_{j_{2}}}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}} & t_{jx} \\\frac{2\left( {{q_{j_{1}}q_{j_{2\;}}} + {q_{j_{0}}q_{j_{3}}}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}} & {1 - \frac{2\left( {q_{j_{1\;}}^{2} + q_{j_{3}}^{2}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}}} & \frac{2\left( {{q_{j_{3}}q_{j_{3\;}}} - {q_{j_{0}}q_{j_{1}}}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}} & t_{jy} \\\frac{2\left( {{q_{j_{1}}q_{j_{3\;}}} - {q_{j_{0}}q_{j_{2}}}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}} & \frac{2\left( {{q_{j_{2}}q_{j_{3\;}}} + {q_{j_{0}}q_{j_{1}}}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{3}}^{2} + q_{j_{3}}^{2}}\end{matrix}} & {1 - \frac{2\left( {q_{j_{2\;}}^{2} + q_{j_{1}}^{2}} \right)}{\begin{matrix}{q_{j_{0}}^{2} + q_{j_{1}}^{2} +} \\{q_{j_{2}}^{2} + q_{j_{3}}^{2}}\end{matrix}}} & t_{jz} \\0 & 0 & 0 & 1\end{bmatrix}} & {{Equation}\mspace{14mu} 5}\end{matrix}$

The processor 405 may further transform 515 the swath output pointprojection coordinates 457 of each lidar point 140 of a texel swath 160into the swath point projection coordinates 453 of the texel swath 160.In one embodiment, the swath output point projection coordinates SPPC457 are transformed 515 into swath point projection coordinates SPC 453using Equation 6, where M_(B) is a transformation matrix employingeither axial or rotational coordinates.SPC=M _(B)SPPC  Equation 6

The processor 405 may transform 520 the swath pixel coordinates 460 ofeach lidar point 140 into output pixel coordinates OPC 230 of a basetexel swath 160. The base texel swath 160 may be selected from theplurality of texel swaths 160. The output coordinate space 315 may bethe coordinate space of the base texel swath 160. The swath pixelcoordinates 460 may be transformed 520 using the swath coordinatetransformation M_(SCT) 225 for the texel swath 160 of the lidar point140 as shown in Equation 7.OPC=M _(SCT)SPC  Equation 7

The processor 405 may identify 525 a swath pixel coordinate that isnearest to a projection of a given lidar point 140 in an associatedoptical image 145. In one embodiment, the processor 405 employs aminimum distance algorithm to identify 525 the image pixel 165.

The processor 405 may transform 530 the swath pixel coordinates IPC 213of the identified given image pixel 165 into the output pixelcoordinates 230 using the swath coordinate transformation M_(SCT) 225 asshown in Equation 8.OPC=M _(SCT)IPC  Equation 8

The processor 405 may identify 535 a matching image pixel 229 in eachoverlapping texel swath 160. The matching image pixel 229 may have theswath output point projection pixel coordinates 457 of the given lidarpoint 140.

The processor 405 may project 540 the swath point projection coordinates253 of each lidar point 140 into the texel swatch 160 with normalizedprojection coordinates. The output pixel coordinates 230 may betransformed 540 using Equation 1.

The processor 405 may remove 545 one or more outlier lidar points 140.The outlier lidar points 140 may be outside of the terrain 124 thetextured digital elevation model 425.

The processor 405 may generate 550 the projection matrix 450. Theprojection matrix 450 may include the lidar point data 255 for eachlidar point 140 of each texel swath 160. The lidar point data 255 mayinclude the normalized projection coordinates 155. In addition, thelidar point data 255 may include the range measurement 290 if the rangemeasurement 290 is available for the lidar point 140.

The processor 405 may identify 555 misregistration errors between thenormalized projection coordinates 155 and calibrated projectioncoordinates 159 as the calibrated projection error 275. In oneembodiment, the normalized projection coordinates 155 are for the firstnormalized point projection 286. The calibrated projection coordinates159 may be for the first matching normalized projection coordinate 231a.

The processor 405 may identify 560 misregistration between thenormalized projection coordinates 155 and matching projectioncoordinates 228 as the matching projection error 280. The normalizedprojection coordinates 155 may be the second normalized point projection289. The matching projection error 280 may be the second matchingnormalized projection coordinate 231 b.

The processor 405 may identify 565 misregistration between the rangemeasurement 290 and a range computed from transformed lidar pointcoordinates as the range error 288. The range may be computed fromtransformed lidar point coordinates may be the second vector 285.

The processor 405 may further calculate 570 the enhanced lidarcoordinates 270 for each lidar point 140 as a function of minimizing themisregistration errors 275/280/288. In one embodiment, themisregistration errors 275/280/288 are minimized using a least-squarescost function. In a certain embodiment, the cost function ε² iscalculated as shown in Equation 9, wherein M is a number of texelswaths, I is a set of lidar points, j is a swath index, i is a lidarpoint within a jth swath, λ_(ij) is a measured range to the first lidarpoint ij, {circumflex over (λ)}_(ij) is an adjusted range to enhancedlidar coordinates for the first lidar point ij, x_(ij) is a firstmatching projection coordinate of the first lidar point ij, y_(ij) is asecond matching projection coordinate of the first lidar point ij,{circumflex over (x)}_(ij) is a first normalized projection coordinateof the first lidar point ij, ŷ_(ij) is a second normalize coordinate ofthe first lidar point ij, σ_(I) ² is a variance of the first rotationalerror, σ_(Ī) ² is a variance of the second rotational error, and σ_(λ) ²is variance of the distance error.

$\begin{matrix}{ɛ^{2} = {{\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{I}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\overset{\_}{I}}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\lambda}^{2}}\left\lbrack \left( {\lambda_{ij} - {\hat{\lambda}}_{ij}} \right)^{2} \right\rbrack}}}}} & {{Equation}\mspace{14mu} 9}\end{matrix}$

The processor 405 may create 575 the enhanced output pixel coordinates235 of each image pixel 165 based on the enhanced lidar coordinates 270for the lidar points 140 and the associated output pixel coordinates230. In one embodiment, the enhanced output pixel coordinates EOPC 235are calculated using Equation 10, where M_(C) is a transformation fromthe output pixel coordinates 230 to the enhanced lidar coordinates 270for the lidar points 140EOPC=M _(C)OPC  Equation 10

The processor 405 may generate 580 the textured digital elevation model425 based on one or more of the enhanced output pixel coordinates 235for each image pixel 165 and the enhanced lidar coordinates 270 and themethod 500 ends. In addition, the textured digital elevation model 425may be generated based on the enhanced output point projectioncoordinates 472. In one embodiment, each output pixel 430 includes thepixel data 435 presented at the enhanced output pixel coordinates 235.As a result, each output pixel 430 is more precisely presented,improving the quality of the textured digital elevation model 425.

The embodiments may use the lidar points 140 to generate the enhancedoutput pixel coordinates 235 and improve the placement of the outputpixels 430 within the textured digital elevation model 425. As a result,the output pixels 430 more precisely render the terrain 120.

The embodiments may be practiced in other specific forms. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. A method comprising: projecting, by use of aprocessor, swath point projection coordinates of each lidar point oftexel swaths into a corresponding texel swath with normalized projectioncoordinates, wherein each texel swath comprises lidar points, an opticalimage, and swath output projection point coordinates of the lidar pointsinto the optical image; generating a projection matrix comprising lidarpoint data for each lidar point of each texel swath, wherein the lidarpoint data comprises the normalized projection coordinates; calculatingenhanced lidar coordinates for each lidar point as a function ofminimizing misregistration errors; creating enhanced output pixelcoordinates for each image pixel based on the enhanced lidar coordinatesfor the lidar points and the associated output pixel coordinates; andgenerating a textured digital elevation model based on the enhancedoutput pixel coordinates for each image pixel, and the enhanced lidarcoordinates.
 2. The method of claim 1, the method further comprising:capturing the texel swaths; transforming lidar point coordinates of eachlidar point from the texel swaths into swath output point projectioncoordinates of an output coordinate space; transforming the swath outputpoint projection coordinates of each lidar point of a given texel swathinto swath pixel coordinates of the given texel swath; transforming theswath pixel coordinates of each lidar point into output pixelcoordinates of a base texel swath using a swath coordinatetransformation for the texel swath of the lidar point; identifying aswath pixel coordinate that is nearest to a projection of a given lidarpoint into an associated optical image; transforming swath pixelcoordinates of the given pixel into the output pixel coordinates usingthe swath coordinate transformation corresponding to the texel swath ofthe given lidar point; and identifying matching image pixels in eachoverlapping texel swath with the swath output point projection pixelcoordinates of the given lidar point.
 3. The method of claim 1, themethod further comprising: identifying the misregistration errorsbetween the normalized projection coordinates and calibrated projectioncoordinates; identifying the misregistration between the normalizedprojection coordinates and matching projection coordinates; andidentifying misregistration between a range measurement and a rangecomputed from transformed lidar point coordinates.
 4. The method ofclaim 3, wherein the misregistration errors comprise a range error thatis a difference of distances between a first vector to first coordinatesof a given lidar point and a second vector to a new coordinates of thegiven lidar point, a calibrated projection error that is a distance onnormalized image plane between first matching projection coordinates ofthe first coordinates in a first optical image captured concurrentlywith the first lidar point and a first point projection that is aprojection of normalized projection coordinates in the first opticalimage, and a matching projection error that is a distance from secondmatching projection coordinates that are a projection of the normalizedprojection coordinates to a second optical image to a second pixelprojection.
 5. The method of claim 4, wherein the misregistration errorsare minimized using a least-squares cost function.
 6. The method ofclaim 5, wherein the cost function ε² is calculated${ɛ^{2} = {{\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{I}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\overset{\_}{I}}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\lambda}^{2}}\left\lbrack \left( {\lambda_{ij} - {\hat{\lambda}}_{ij}} \right)^{2} \right\rbrack}}}}},$wherein M is a number of texel swaths, I is a set of lidar points, j isa swath index, i is a lidar point within a jth swath, λ_(ij) is ameasured range to the first lidar point ij,{circumflex over (λ)}_(ij) isan adjusted range to enhanced lidar coordinates for the first lidarpoint ij, x_(ij) is a first matching projection coordinate of the firstlidar point ij, y_(ij) is a second matching projection coordinate of thefirst lidar point ij, {circumflex over (x)}_(ij) is a first normalizedprojection coordinate of the first lidar point ij, ŷ_(ij) is a secondnormalized coordinate of the first lidar point ij, σ_(I) ² is a varianceof the first rotational error, σ_(Ī) ² is s a variance of the secondrotational error, and σ_(λ) ² is variance of the distance error.
 7. Themethod of claim 1, wherein each lidar point is fused to thecorresponding optical image at a calibrated pixel location.
 8. Anapparatus comprising: a processor; a memory storing code executable bythe processor to perform: projecting swath point projection coordinatesof each lidar point of texel swaths into a corresponding texel swathwith normalized projection coordinates, wherein each texel swathcomprises lidar points, an optical image, and swath output projectionpoint coordinates of the lidar points into the optical image; generatinga projection matrix comprising lidar point data for each lidar point ofeach texel swath, wherein the lidar point data comprises the normalizedprojection coordinates; calculating enhanced lidar coordinates for eachlidar point as a function of minimizing misregistration errors; creatingenhanced output pixel coordinates for each image pixel based on theenhanced lidar coordinates for the lidar points and the associatedoutput pixel coordinates; and generating a textured digital elevationmodel based on the enhanced output pixel coordinates for each imagepixel, and the enhanced lidar coordinates.
 9. The apparatus of claim 8,the processor further performing: capturing the texel swaths;transforming lidar point coordinates of each lidar point from the texelswaths into swath output point projection coordinates of an outputcoordinate space; transforming the swath output point projectioncoordinates of each lidar point of a given texel swath into swath pixelcoordinates of the given texel swath; transforming the swath pixelcoordinates of each lidar point into output pixel coordinates of a basetexel swath using a swath coordinate transformation for the texel swathof the lidar point; identifying a swath pixel coordinate that is nearestto a projection of a given lidar point into an associated optical image;transforming swath pixel coordinates of the given pixel into the outputpixel coordinates using the swath coordinate transformationcorresponding to the texel swath of the given lidar point; andidentifying matching image pixels in each overlapping texel swath withthe swath output point projection pixel coordinates of the given lidarpoint.
 10. The apparatus of claim 8, the processor further performing:identifying the misregistration errors between the normalized projectioncoordinates and calibrated projection coordinates; identifying themisregistration between the normalized projection coordinates andmatching projection coordinates; and identifying misregistration betweena range measurement and a range computed from transformed lidar pointcoordinates.
 11. The apparatus of claim 10, wherein the misregistrationerrors comprise a range error that is a difference of distances betweena first vector to first coordinates of a given lidar point and a secondvector to a new coordinates of the given lidar point, a calibratedprojection error that is a distance on a normalized image plane betweenfirst matching projection coordinates of the first coordinates in afirst optical image captured concurrently with the first lidar point anda first point projection that is a projection of normalized projectioncoordinates in the first optical image, and a matching projection errorthat is a distance from second matching projection coordinates that area projection of the normalized projection coordinates to a secondoptical image to a second pixel projection.
 12. The apparatus of claim11, wherein the misregistration errors are minimized using aleast-squares cost function.
 13. The apparatus of claim 12, wherein thecost function ε² is calculated${ɛ^{2} = {{\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{I}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\overset{\_}{I}}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\lambda}^{2}}\left\lbrack \left( {\lambda_{ij} - {\hat{\lambda}}_{ij}} \right)^{2} \right\rbrack}}}}},$wherein M is a number of texel swaths, I is a set of lidar points, j isa swath index, i is a lidar point within a jth swath, λ_(ij) is ameasured range to the first lidar point ij, {circumflex over (λ)}_(ij)is an adjusted range to enhanced lidar coordinates for the first lidarpoint ij, x_(ij) is a first matching projection coordinate of the firstlidar point ij, y_(ij) is a second matching projection coordinate of thefirst lidar point ij, {circumflex over (x)}_(ij) is a first normalizedprojection coordinate of the first lidar point ij, ŷ_(ij) is a secondnormalized coordinate of the first lidar point ij, σ_(I) ²lis a varianceof the first rotational error, σ_(Ī) ² is s a variance of the secondrotational error, and σ_(λ) ² is variance of the distance error.
 14. Theapparatus of claim 8, wherein each lidar point is fused to thecorresponding optical image at a calibrated pixel location.
 15. Aprogram product comprising a non-transitory computer readable storagemedium storing code that is executable by a processor to perform:projecting swath point projection coordinates of each lidar point oftexel swaths into a corresponding texel swath with normalized projectioncoordinates, wherein each texel swath comprises lidar points, an opticalimage, and swath output projection point coordinates of the lidar pointsinto the optical image; generating a projection matrix comprising lidarpoint data for each lidar point of each texel swath, wherein the lidarpoint data comprises the normalized projection coordinates; calculatingenhanced lidar coordinates for each lidar point as a function ofminimizing misregistration errors; creating enhanced output pixelcoordinates for each image pixel based on the enhanced lidar coordinatesfor the lidar points and the associated output pixel coordinates; andgenerating a textured digital elevation model based on the enhancedoutput pixel coordinates for each image pixel, and the enhanced lidarcoordinates.
 16. The program product of claim 15, the processor furtherperforming: capturing the texel swaths; transforming lidar pointcoordinates of each lidar point from the texel swaths into swath outputpoint projection coordinates of an output coordinate space; transformingthe swath output point projection coordinates of each lidar point of agiven texel swath into swath pixel coordinates of the given texel swath;transforming the swath pixel coordinates of each lidar point into outputpixel coordinates of a base texel swath using a swath coordinatetransformation for the texel swath of the lidar point; identifying aswath pixel coordinate that is nearest to a projection of a given lidarpoint into an associated optical image; transforming swath pixelcoordinates of the given pixel into the output pixel coordinates usingthe swath coordinate transformation corresponding to the texel swath ofthe given lidar point; and identifying matching image pixels in eachoverlapping texel swath with the swath output point projection pixelcoordinates of the given lidar point.
 17. The program product of claim15, the processor further performing: identifying the misregistrationerrors between the normalized projection coordinates and calibratedprojection coordinates; identifying the misregistration between thenormalized projection coordinates and matching projection coordinates;and identifying misregistration between a range measurement and a rangecomputed from transformed lidar point coordinates.
 18. The programproduct of claim 17, wherein the misregistration errors comprise a rangeerror that is a difference of distances between a first vector to firstcoordinates of a given lidar point and a second vector to a newcoordinates of the given lidar point, a calibrated projection error thatis a distance on a normalized image plane between first matchingprojection coordinates of the first coordinates in a first optical imagecaptured concurrently with the first lidar point and a first pointprojection that is a projection of normalized projection coordinates inthe first optical image, and a matching projection error that is adistance from second matching projection coordinates that are aprojection of the normalized projection coordinates to a second opticalimage to a second pixel projection.
 19. The program product of claim 18,wherein the misregistration errors are minimized using a least-squarescost function.
 20. The program product of claim 19, wherein the costfunction ε² is calculated${ɛ^{2} = {{\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{I}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\overset{\_}{I}}^{2}}\left\lbrack {\left( {x_{ij} - {\hat{x}}_{ij}} \right)^{2} + \left( {y_{ij} - {\hat{y}}_{ij}} \right)^{2}} \right\rbrack}}} + {\underset{j = 0}{\sum\limits^{M - 1}}{\sum\limits_{i \notin I_{j}}{\frac{1}{\sigma_{\lambda}^{2}}\left\lbrack \left( {\lambda_{ij} - {\hat{\lambda}}_{ij}} \right)^{2} \right\rbrack}}}}},$wherein M is a number of texel swaths, I is a set of lidar points, j isa swath index, i is a lidar point within a jth swath, λ_(ij) is ameasured range to the first lidar point ij, {circumflex over (λ)}_(ij)is an adjusted range to enhanced lidar coordinates for the first lidarpoint ij, x_(ij) is a first matching projection coordinate of the firstlidar point ij, y_(ij) is a second matching projection coordinate of thefirst lidar point ij, {circumflex over (x)}_(ij) is a first normalizedprojection coordinate of the first lidar point ij, ŷ_(ij) is a secondnormalized coordinate of the first lidar point ij, σ_(I) ² is a varianceof the first rotational error, σ_(Ī) ² is a variance of the secondrotational error, and σ_(λ) ² is variance of the distance error.